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In response to the increased targets in the 2021 Levels plan, the Department has developed an Advanced 
Analytics (AA) model to assist in the processing of applications in the Spouse or Common-Law Partner in 
Canada Class. The model uses a variety of criteria to assess the eligibility of both the sponsor and the 
principal applicant (PA.) Based on its assessment the model assigns applications to either a "Green" or a 
"Standard" bin. Applications in the Green Bin receive a positive administrative eligibility determination 
based solely on the model's recommendation, while Standard Bin applications will be processed by 
Officers as per normal procedures. For all applications, a human officer is responsible for assessing the 
eligibility of dependents, the admissibility of the PA and dependents, and for taking the final decision. 


The model's overall impact on clients is expected to be minimal given that the model neither refuses 
applications nor makes any form of negative recommendation, but rather assigns applications into one of 
the two bins. Accordingly, the "worst" potential outcome for any application passed through the model 
would be that it is processed by an officer as per normal procedures. 


= yti Ca 


This analysis proceeds primarily by comparing, along different dimensions, the proportion of applications 
in an inventory before it is run through the model to the proportion of applications in the Green and 
Standard bins after assessment, and noting any significant variations, or lack thereof. 


For example, if same-sex partnerships account for 20% of the inventory fed into the model, but then 
account for only 5% of the Green Bin, this could indicate that the model may have introduced an 
unintended bias against same-sex couples. Of course, this is not necessarily so. There may be something 
in the initial inventory that explains and justifies the discrepancy. 


Furthermore, each triage criterion is explained and 
justified in the Officer of Record memo to assign accountability of model decisions and to prevent any 
unintended, differential distribution of triage results. 


The initial analysis was conducted on the open applications that were triaged by the model for the first 
time, most of which were backlogs. The set comprised of approx. 6,000 open applications in the SCLPC 
inventory when the model was deployed initially. In June 2021, a minor update was made to the model 
to enhance the program integrity. In order to monitor the model impact to identify and mitigate any 
unintended outcomes, a follow-up analysis was conducted by following the same method in March 2022, 
when the number of triaged applications by the model reached approx. 34, 000. 
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4.1 Model Development and Testing Data 
- Gender Dimensions of the Green versus Standard Bin 


The data set used to develop and test the AA model contained 40,343 applications, 23,555 (58.496) of 
which had female PAs, while 16,785 (41.696) had male PAs. When these applications were run through 
the model, it passed 23,942 into the Green Bin and 16,401 into the Standard Bin. 


Relatedly, of the 40,343 applications in the development and testing inventory, the sponsor and the PA 
were of a different sex in 38,747 (96%) applications, while 1,596 (496) were of the same gender. 
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4.2 Model Deployment — Initial Analysis 


- Gender Dimensions of the Green versus Standard Bin 


The model was run over a set of open applications for the first time on April 27^, 2021. The set 
comprised 6,046 open applications in the SCLPC inventory. In 3,380 (55.996) of the 6,046 applications the 
PA was female, while 2,666 (44.196) had male PAs; 3,483 (57.696) were assigned into the Green Bin and 
2,563 (42.496) into the Standard Bin. 
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4.3 Model Deployment — Follow-up Analysis for Monitoring 


- Gender Dimensions of the Green versus Standard Bin 


By mid-March 2022, the model has triaged 34,018 SCPLC applications, of which 18,418 (54.1%) 
applications had female PAs, while 15,574 (45.896) had male PAs. Among these triaged applications, 
17,640 were assigned into the Green Bin and 16,378 into the Standard Bin. 


Female — 00 ss 18418 s MEME c c 


Male s 15,974 — 45.896 — s 
l— Other 7 00% 0 0 | 
Unknown | 0 19 s 01% 000 | 

All 34,018 100.0% 
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Based on the initial and follow-up analysis, data from testing and implementation stage indicate that the 
model does not currently appear to have any significant, disproportionate impact on clients. Assignments 
into the Green and Standard bins appear relatively equitable, with notable variances easily understood 
following review of the data. 


In terms of gender profile, both the individual applicants and their relationships in the Green and Standard 
bins closely matches the related profiles in the testing dataset and model implementation for both initial 
and follow-up analysis, thus the model did not result in any differential output along the dimension of 
gender. 


An examination of country-of-birth data revealed that the model eligibility approval rate is being evenly 
‘distributed’ to most countries. In the case of a few countries, where more notable variations were 
found, a closer examination of the data showed that applications from those countries more commonly 
failed to meet certain model rules. As mentioned in the earlier section of this analysis, the model's 
overall impact on clients is expected to be minimal because the model neither refuses applications nor 
makes any refusal recommendations, but rather assigns applications into one of the two bins. Therefore, 
the applications that were triaged in the standard bin with PAs born in these countries are processed by 
an officer as per normal procedures, which may still result with an approved final decision. 


It is observed that no unintended bias is immediately apparent in the model’s initial output and ongoing 
deployment. The overall potential for any negative impacts on specific groups of applicants is mitigated 
in large part by the fact that the model only automates certain positive eligibility decisions and that all 
final decisions are always made by an officer. 
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